Intelligent internet system with adaptive user interface providing one-step access to knowledge

ABSTRACT

An Intelligent Internet system having the capacity to discover, acquire, organize, store, and make accessible knowledge about things of interest to users and to help users more efficiently interact with these things in a rich variety of ways. The Intelligent Internet system provides users with a personalized interface that dynamically adapts to each user&#39;s interests and context. This interface provides users with direct access to “knowledge” about things that humans or machines can uniquely sense, recognize, name, understand and interact with, whether real or imagined. Users can access the knowledge stored in the Intelligent Internet system to find, learn about and select products and services; communicate with family, friends, colleagues, classmates, customers and suppliers; learn about things they encounter in their environment, make decisions about them, consume and share content related to them and perform transactions.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a continuation of U.S. patent applicationSer. No. 14/212,654, entitled INTELLIGENT INTERNET SYSTEM WITH ADAPTIVEUSER INTERFACE PROVIDING ONE-STEP ACCESS TO KNOWLEDGE, filed Mar. 14,2014, which claims benefit to U.S. provisional patent application No.61/800,302, entitled INTELLIGENT INTERNET SYSTEM WITH ADAPTIVE USERINTERFACE PROVIDING ONE-STEP ACCESS TO KNOWLEDGE, filed on Mar. 15,2013. The entire contents of which are incorporated herein by referencefor all purposes.

FIELD OF THE INVENTION

The disclosed embodiments relate generally to digital assistants, andmore specifically to digital assistants that intelligently provideassistance to help users learn about and interact with things.

BACKGROUND AND SUMMARY

All human activities involve interacting with “entities,” i.e., thingsreal and imagined in our internal and external environments. Examples ofentities include: people, places, organizations, companies, restaurants,movies, books, products, ideas, beliefs, theories, events . . .thousands of types of things we can uniquely perceive, understand, andname. Only a few of these entities are currently directly addressable bythe Internet. The vast majority is not.

As people interact with entities, the human brain acquires knowledgeabout them, allowing future interactions to be more efficient. Some ofthis knowledge is passed on to future generations. Thus, there are twowell-known repositories of encoded knowledge in humans: DNA (whichencodes knowledge inherited from our ancestors) and memories (whichencode knowledge we gain through our personal interactions with entitiesin the form of cellular changes to neurons in the brain). Theseknowledge structures help us interact with things in our environmentmore efficiently.

As described below, the Intelligent Internet system, according to oneexample, augments these two repositories, by providing a virtualnetworked system with the capacity to acquire, encode, store, access,use and share many types of collective knowledge known to humans andmachines.

The Internet was originally designed as a network of networks, amechanism for connecting computer servers, devices called terminals,content files and software applications. Each device connected by theInternet is assigned its own “Internet Protocol Address” (IP address), anumerical label assigned, often dynamically, to each deviceparticipating in computer networks that use the Internet Protocol forcommunication. Over time, the Internet has evolved to connect billionsof devices, each such device having its own assigned IP address and eachable to communicate efficiently with other devices having assigned IPaddresses via the Internet.

Today's Internet has two major deficiencies: 1) it lacks an innateability to acquire and apply knowledge about the entities it connects,and 2) it does not allow users to connect directly to and interact withthe vast majority of entities currently lacking assigned IP addresses,for example: people, places, events and other things the human brain isable to recognize, understand and interact with. These deficienciescurrently lack a commercially viable solution.

As a result, there is a major disconnect between a system that knowsonly how to locate and communicate with things having assigned IPaddresses (devices, files and applications) and a fundamental need ofmany users to locate, connect with, learn about and interact with thingsof interest to them, most of which do not have assigned IP addresses.Today's Internet and the human brain essentially speak two differentlanguages.

Thus far, the World Wide Web has been the most successful attempt tobridge this disconnect. The Web essentially allows us to create2-dimensional “hypertext” connections between a tagged object in a fileand a related addressable file, and very importantly the hypertext tagsare capable of pointing only to a single IP address or a URL associatedwith an IP address. However, the Web only understands “files” while thehuman brain understands “things,” i.e. entities. Both the Internet andthe Web speak languages difficult for the human brain to decipher. TheWeb places upon its users the burden of making sense of the files withwhich it is capable of connecting. It only helps users connect with“things” that have assigned IP addresses. Once connected with these“things” users must do most of the work thereafter.

The Web has a number of other important deficiencies related to controland risk. Hypertext links are controlled by website owners, not users.Users who click on or touch a hyperlinked object typically have littleidea where they may be taken, and indeed may end up accessing dangerousfiles. The World Wide Web paradigm has resulted in the creation ofhundreds of millions of websites controlled by owners and operators ofunknown reputation, each website having its own custom-made userinterface.

A more recent paradigm called the “Semantic Web” has partially addressedthe inability of the Web to understand “things.” The Semantic Weborganizes and stores two major kinds of knowledge (facts andrelationships) about things in file structures that computers canunderstand. The Semantic Web generally uses a file structure called“Resource Description Framework” (RDF) which decomposes knowledge intosmall pieces. Today there are billions of RDF records from resourcessuch as Wikipedia and The New York Times that contain snippets ofknowledge accessible by devices connected to the Internet.

However, the Semantic Web has its own major deficiencies, including: 1)most importantly, like the Internet and World Wide Web, it lacks aninnate capacity to acquire and apply knowledge; 2) it is designed toenable computers to understand its content, not humans; and 3) it storesonly a few simple types of the knowledge that we humans consider usefulor even essential for interacting with things of interest to us in ourenvironment.

A widely accessible definition of “Intelligence” is: “the capacity toacquire and apply knowledge.” (See, e.g., American Heritage Dictionaryof the English Language.) As described herein, the Intelligent Internetsystem, according to one example, augments the capabilities of today'sInternet by creating a networked system with the capacity to acquire,organize, encode, store, access, apply and share many types of knowledgeabout entities currently connected by the Internet, as well as entitiesnot currently connected by the Internet.

The Intelligent Internet system, according to one example, has thecapacity to directly acquire knowledge about human behaviors because itis a single unified system that allows users to interact directly withentities. It organically integrates the user interface, userinteractions with entities, and an operating system having access toknowledge and resources, eliminating complex junction points betweenthem. Alternate paradigms such as the World Wide Web, the Semantic Weband Mobile Apps all have complex junction points between the userinterface, applications, operating system and various chaotic resources.

Entities can be difficult for both humans and machines to recognize anddisambiguate. A major challenge in recognizing and disambiguating amongentities is the fact that a specific entity may be represented bydiffering “symbolic objects.” These symbolic objects appear everywherein our environment: on display screens of devices, in things we see withour eyes or hear with our ears. They can include names (“Tim BernersLee”), nicknames, aliases, images, videos, IDs, codes. They can appearin hundreds of types of computer readable content, such as news, tweets,messages, quotations, etc.

A user of the Intelligent Internet system, in one embodiment, candirectly learn about these symbolic objects and the entities theyrepresent without the aid of the World Wide Web, for example, by:selecting or specifying an entity of interest using a Smart Phone,tablet, PC, Smart TV or other computing and communications devices or bycapturing images or videos of entities of using digital cameras or otherimaging devices; etc.

In one example, the Intelligent Internet organizes, encodes and storesknowledge about entities in a new type of pol-dimensional digitalstructure referred to as “knowledge cells.” Each of these knowledgecells represents the intersection of at least an entity and a contentobject. For example, a knowledge cell may represent the intersection ofa single person, place, company, event, belief or other type of entityand a single content object such as a news article, a tweet or othertype of content. In this example, the Intelligent Internet implementsknowledge cells as poly-dimensional digital structures, each of whichencodes a rich set of knowledge about a specific entity, including:factual knowledge about this entity; knowledge about relationshipsbetween this entity and other entities; knowledge about valid methodsfor interacting with this entity by humans and machines; knowledge aboutratings, reviews and opinions; knowledge providing guides, guidelinesand procedures for interacting with this entity.

In one embodiment, the encoded factual knowledge includes propertiesclassified as “knowledge markers, which collectively have the capacityto uniquely describe an entity, thus allowing humans and machines tomore efficiently disambiguate among entities.

In one example of the current invention, the Intelligent Internetprovides a rich library of “Smart Interaction Modules” (“SIMs”),software that implements standardized methods for interacting withentities. Users accessing the Intelligent Internet with touch-enabledmobile devices or tablets can directly access these Smart InteractionModules simply by touching any symbolic object wherever they encounterit . . . in a news article, a tweet, an image, a video or on a website.Users accessing the Internet with other types of devices can directlyaccess these Smart Interaction Methods by using a point and clickmechanism or by speaking to select an entity or specify it.

One aspect of the current invention implements a new unified paradigm tohelp users interact with entities of personal interest to them,including, for example: 1) a uniform scheme for uniquely identifying alltypes of entities (e.g. unique entity IDs); 2) an efficientmulti-dimensional structure for encoding, organizing and storing ourcollective knowledge about entities; 3) one-step access to usefulknowledge about entities of personal interest; 4) a standardized libraryof smart methods for interacting with entities; 5) a simplified way tolearn about updates concerning entities; and 6) a simple, adaptive userinterface that understands users, their context and the entities ofinterest to them, and adapts dynamically to help users interact withthese entities and 7) a simplified, more efficient and secure way tocommunicate messages of all types.

In one aspect and example, the intelligent Internet system is auniversal networked system that acquires, accumulates and efficientlystores knowledge about entities and symbolic objects representing theseentities, and makes this knowledge easily available to users.

In another aspect and example, the Intelligent Internet system has thecapacity to acquire and accumulate knowledge about entities fromnumerous diverse resources, including: news feeds, tweets, Web Services,Open Linked Content, Web sites, digital publications, computer files,office documents and user behaviors and inputs.

In another aspect and example, the Intelligent Internet system, enablesusers to directly access knowledge about a symbolic object they see onthe screen of a Smart Phone, tablet, PC or other Smart Device, simply bytouching the symbolic object.

BRIEF DESCRIPTION OF THE DRAWINGS

The present application can be best understood by reference to thefollowing description taken in conjunction with the accompanying drawingfigures, in which like parts may be referred to by like numerals.

FIG. 1 shows a block diagram of a first embodiment of the inventionhaving methods for acquiring, managing, storing and making accessiblecontent and knowledge.

FIG. 2 shows a block diagram of an embodiment having methods for storingmetadata components.

FIG. 3 shows an embodiment having methods for obtaining content andservices from multiple types of content resources.

FIG. 4 shows an embodiment having methods for rationalizing contentresources and making them available to the Intelligent Internet system.

FIG. 5 shows a block diagram of an embodiment having methods forprocessing RSS content resources.

FIG. 6 shows a block diagram of an embodiment having methods to RDFcontent resources.

FIG. 7 is a block diagram showing an embodiment, having methods forprocessing content from Web Services resources.

FIG. 8 shows a block diagram of an embodiment having methods to processWeb pages content resources.

FIG. 9 shows a block diagram of an embodiment having methods forprocessing office documents content resources.

FIG. 10 shows a block diagram of an embodiment having methods forrecognizing symbolic objects and adding knowledge tags to symbolicobjects.

FIG. 11 shows a block diagram of an embodiment having methods forrecognizing and processing symbolic objects embedded in text contentresources.

FIG. 12 shows a block diagram of an embodiment having methods forrecognizing and processing symbolic objects embedded in image contentresources.

FIG. 13 shows a block diagram of an embodiment having methods forrecognizing and processing symbolic objects embedded in video contentresources.

FIG. 14 shows a block diagram of an embodiment having methods forrecognizing and processing symbolic objects embedded in video broadcastcontent resources.

FIG. 15 shows a block diagram of an embodiment having methods processingfor recognizing symbolic objects embedded in printed content resources.

FIG. 16 is a flow chart showing an embodiment having methods foracquiring explicit factual knowledge about entities.

FIG. 17 is a flow chart showing an embodiment having methods foracquiring knowledge about explicit relationships among entities.

FIG. 18 is a flow chart showing an embodiment having methods foracquiring contextual knowledge about relationships among entities.

FIG. 19 is a flow chart showing an embodiment having methods formonitoring and processing user interactions with entities.

FIG. 20 shows a block diagram of an embodiment having methods forprocessing and storing knowledge contributed by users.

FIG. 21 shows a block diagram of an embodiment having methods fororganizing metadata for smart content records.

FIG. 22 shows a block diagram of an embodiment having methods forimplementing metadata-defined elements and structures in a knowledgegraph.

FIG. 23 shows an embodiment having functions managing the creation ofentity nodes based on “entity type” node templates.

FIG. 24 shows an embodiment having functions to optimize searching forknowledge about entities in a knowledge graph.

FIG. 25 is a block diagram showing an embodiment having methods forprocessing smart messages.

FIG. 26 is a block diagram showing an embodiment having methods forprocessing messages from message sources.

FIG. 27 is a block diagram showing an implementation having methods forpartitioning smart messages into smart message components.

FIG. 28 is a block diagram showing an embodiment having methods forstoring smart message summaries in an iPv6-addressable storage system.

FIG. 29 is a block diagram showing methods for implementing an alternateembodiment storing smart message summaries in an URI-based system.

FIG. 30 is a block diagram showing me an embodiment having methods forsubscribing to smart message summaries.

FIG. 31 shows a block diagram showing an embodiment having methods to“Follow” smart messages about any entity.

FIG. 32 is a block diagram showing an embodiment having methods allowingusers and suppliers to communicate one-to-one in real time.

FIG. 33 shows a block diagram of an embodiment having methods forselecting entities for interactions.

FIG. 34 is a block diagram showing an embodiment having methods tosearch for entities, assisted by entity knowledge markers.

FIG. 35 is a block diagram showing an embodiment having methods forselecting an entity of Interest from among a list of symbolic objects.

FIG. 36 is a block diagram showing an embodiment having methods forselecting an entity from among a gallery of symbolic objects.

FIG. 37 is a block diagram showing an embodiment having methods forselecting a tagged symbolic object embedded in smart text content.

FIG. 38 is a block diagram showing an embodiment having methods forselecting an untagged symbolic object embedded in smart text content.

FIG. 39 is a block diagram showing an embodiment having methods forselecting an untagged symbolic object embedded in an image object.

FIG. 40 a block diagram showing an embodiment having methods forselecting an untagged symbolic object embedded in printed content.

FIG. 41 is a block diagram showing an embodiment having methods forselecting an untagged symbolic object embedded in video content.

FIG. 42 is a flow chart showing an implementation having methods foraccessing knowledge about applicable interactions for entities.

FIG. 43 is a block diagram showing an embodiment having methods tomanage user interactions with entities, starting with the user selectingan entity.

FIG. 44 is a block diagram showing an embodiment having methods tomanage user interactions with entities, starting with the user selectingan interaction method.

FIG. 45 is a decision matrix showing an embodiment having methods formanaging user interactions with entities.

FIG. 46 is an illustration showing an embodiment having methods forusing an icon to select a smart content record for interaction.

FIG. 47 is a block diagram showing examples of an implementation havingmethods for interacting with smart content.

FIG. 48 is a block diagram showing an implementation having methods forinteracting with entities using “smart interaction modules” (“SIMs”).

FIG. 49 illustrates a design showing an implementation having methodsfor managing an adaptive user interface responsive to user selections ofa symbolic object and/or an entity.

FIG. 50 illustrates a design showing an implementation having methodsfor managing an adaptive user interface on one page.

FIG. 51 illustrates a logical flow chart showing an implementationhaving methods for managing an adaptive user interface responsive touser selection of a universal interaction.

DETAILED DESCRIPTION OF THE INVENTION

The following description is presented to enable a person of ordinaryskill in the art to make and use the various embodiments. Descriptionsof specific devices, techniques, and applications are provided only asexamples. Various modifications to the examples described herein will bereadily apparent to those of ordinary skill in the art, and the generalprinciples defined herein may be applied to other examples andapplications without departing from the spirit and scope of the presenttechnology. Thus, the disclosed technology is not intended to be limitedto the examples described herein and shown, but is to be accorded thescope consistent with the claims.

Embodiments of the present invention provide users with simplifiedaccess to knowledge about “entities.” Entities include all things humanscan uniquely recognize, name and interact with, including for example:people, places, companies, schools, restaurants, events, ideas and muchmore. Entities may include everything one can see, hear, touch, feel orimagine.

Embodiments allow users to find and select entities of interest andinteract with them, using “smart interaction modules” (“SIMs”), modular,integrated computer-implemented methods for interacting with entities.

Embodiments of the invention discover, organize, store, manage, utilizeand make accessible multiple types of knowledge from multiple sourcesabout multiple types of entities.

These embodiments have methods for creating and managing a specificgroup of entities called “content,’ such content comprised ofcollections of words, images and auditory and visual media humans canexperience, interact with and consume. Examples of content entitiesinclude, for example: news articles, tweets, blogs, office documents,eBooks, images and videos. Content entities may be complex structures,and always relate to one or more instances of unitary entities.

Any entity may have “symbolic objects” that represent it (for example,the entity “Abraham Lincoln” can be represented by a written or spokenword or phrase, a nick name, an ID, a code, an image or a video, etc.).

In an embodiment of the invention, users can select and interact with anentity of Interest simply by touching any symbolic object representingthat entity. In other embodiments, users can click on or speak the nameof a symbolic object or entity to select it for further interactions.These embodiments provide users with a simple one step process foraccessing knowledge about entities and provide users a “Touch . . . andKnow” or “Speak . . . and Know” experience.

An embodiment allows users to select entities of Interest by browsinglists or galleries of symbolic objects, each symbolic objectrepresenting an entity, and then selecting a symbolic object of interestfrom the list or gallery.

An embodiment allows users to select an entity by entering a searchrequest in a user interface and interacting with Intelligent Agents. Asthe user begins entering a search request, methods activate anIntelligent Search Agent that starts analyzing the search request andinitiates direct bi-directional communications with a “knowledge graph,”a networked database having encoded, sharable knowledge about entities.If the Intelligent Search Agent determines that the knowledge graphcontains more than one entity meeting the user search requestparameters, it opens a visual panel in the user Interface and displays alist of such possible entities, along with knowledge about them,including: the names of the entities, and for each of the named entitiesadditional knowledge about the entity, including its “entity group,” forexample “Person,” denoting a broad category for the possibly matchingentity; its “entity type,” for example, “Actor,” denoting a more narrowcategory for the possibly matching entity; and “knowledge markers”,specific properties that, when viewed by the user, can help the userselect a specific entity of Interest. Enabled to see such specificknowledge about each possible entity meeting their search request, userscan disambiguate among them and select their specific entity of interestby touching it or clicking on it or speaking its name.

An embodiment allows a user to select a symbolic object embedded in anycontent displayed in the user interface or presented to the user viaspoken words. Selecting any such symbolic object activates an“Intelligent Interaction Agent” that has direct-networked access toknowledge in the knowledge graph about the selected symbolic object, andthe entity it represents. The Intelligent Interaction Agent, whenactivated, automatically performs several tasks for the user, including:retrieving knowledge about valid methods by which the user can interactwith this entity, presenting a menu in the user interface from which theuser can select how they wish to interact with this entity, andpopulating the menu with valid interaction options applying to thisspecific entity.

An embodiment includes a large library of “Smart Interaction Modules”(SIMs), that allow users, for example, to: read and interact with news;upload, organize, manage, view and interact with images and videos; readand interact with eBooks; view and interact with office documents;interact with a restaurant to reserve a table; find an event andinteract with it; post a tweet; receive and view tweets about entitiesof interest and interact with them; post an update to a social network;receive, view and interact with updates from a social network and manyother types of interactions with entities.

Embodiments of the invention have utility for supporting a wide range ofuser activities, including: learning about things; watching TV andinteracting with people, places, events and ideas shown on a TV program;listening to a radio station and interacting with anything spoken orplayed; planning travel; buying and selling things; obtaining andproviding personal and business services; investing and managinginvestment; creating, sending, receiving and interacting with messagesabout anything; managing and interacting with network connected devicesand responding to the signals they create and transmit; analyzing “bigdata” resulting from methods that integrate content, interactions andanalytic capabilities; and many more activities.

In an embodiment of the invention, the user interface allowing users toselect and interact with entities consists of a single page, havingpanels that dynamically adapt to bring the user knowledge, content andother resources he or she seeks.

FIG. 1 shows a block diagram of a first embodiment of the inventionhaving methods for acquiring, managing, storing and making accessiblecontent and knowledge. All components in the invention are connected ina large network.

In an embodiment, “metadata” 1 is stored in a database such as My SQL indata structures that insure its consistency and integrity. The metadataincludes lists 9 and tables 10 helpful to or necessary for categorizingand managing knowledge and “smart content.”

In an embodiment, methods pre-process a variety of content resources 2wherein said methods transform such content resources into “smartcontent” 6 i.e. coherent rationalized content that contains embeddedknowledge about itself in a specific component of each smart contentrecord.

In an embodiment, users interact with variables stored in the knowledgegraph and the smart content database using a touch screen enabled devicehaving a user interface that adapts responsive to: user inputs;variables stored in the user's device or available to the user's device,such as geo-location, movement, orientation, etc.; knowledge about theinterests of a user stored in the knowledge graph and in the user'sdevice; and knowledge about the past interactions of the user withvarious entities. The system encodes these user interactions 3 asbehavioral knowledge in the knowledge graph.

In an embodiment, users contribute various types of knowledge to theIntelligent Internet system by directly contributing inputs to thesystem 4. Users may contribute many types of knowledge individually andcollectively to the Intelligent Internet system, including, for example:questions and answers, opinions, how to's, procedures, guidelines,reactions both positive and negative, specifications of items orservices they want or have to offer. The Intelligent Internet system mayevaluate this knowledge, and correlate it to other knowledge to helpestablish its credibility. For example, the user may contribute a reviewabout a restaurant, and the system may correlate this to knowledgestored elsewhere in the knowledge graph indicating the user has visitedthis restaurant, along with the number of times visited.

In an embodiment, methods pre-process content from content resources 2and transform the content into smart content that it then stores in thesmart content database 6, such methods including: standardizing andnormalizing file types of the content, converting various file typesinto “standardized file types” efficiently supported by the system;standardizing and normalizing names for content properties; andstandardizing and normalizing data types using data conversion methods,such as are implemented in various commercial open source tools asSpring data, into standard data types.

In this embodiment, methods convert resource file types, for example,PDF formatted files, into standard file types well understood by thesystem, such as HTML, XML, ePub3 or other widely used, easily managedand easily displayable standard file formats.

In an embodiment, methods convert Microsoft Office resource files, suchas Word, Excel and PowerPoint files, into standard file types wellunderstood by the smart content database 6, into file types such as,ePub3.

In an embodiment, methods create “smart content” records and store theserecords in an enhanced version of a commercial or open source documentdatabase, such as MongoDB. This embodiment supports the storing of smartcontent in either structured or unstructured formats, or any mixthereof.

In an embodiment, methods transform content into “smart content” byorganizing content records into smart content records, each such smartcontent record having embedded record components that include knowledgeabout the content, including: 1) a component having content metadataknowledge, 2) a component having content knowledge maps, 2) a componenthaving content permissions knowledge, 3) a component having one or morecontent summaries and 4) a component comprised of the original contentobjects from content sources.

An example of content metadata knowledge component embedded in a smartcontent record may include such data as: Title of the content, author(s)of the content, Publication date of the content, publisher of thecontent and editor(s) of the content.

An example of a content knowledge map component embedded in a smartcontent record may include, for example: 1) a computer-generated uniqueentity ID (i.e. a unique random ID applying only to this contentrecord); 2) an entity name (for example, the title of the content); 3)an entity group name (the default value for all smart content is“content”); 4) an entity type name (consisting of the name of a validcontent type as listed in the metadata); 5) an author name for eachauthor of the content; 6) a unique entity ID for each author of thecontent; 7) a unique entity ID for the resource supplying the content;and 8) a “timestamp-published” property recording the date and time thecontent was published, etc.

An example of a content permissions knowledge component embedded in asmart content record may in include such data as: 1) a“timestamp-authorized-release” content property, designating the dateand time the content may first be released; 2) a “timestamp-end-of-life”content property designating the date and time further access shall beblocked; and 3) the entity ID of any user(s) or groups of usersauthorized to access this smart content record.

Examples of a content summaries component in the smart content recordmay include an “alert summary,” consisting of a knowledge map component,knowledge permissions component and a summary of the content component,which may consist, for example, of 100 characters, sufficient for a userto get the gist of what the full content is about.

An example of a content object component in a smart content record wouldbe the original content from the content resources 2, and such contentmay be enhanced to include embedded hidden knowledge tags containing amethod for accessing knowledge in the knowledge graph about knownsymbolic objects in the content object.

In an embodiment, functions combine the capabilities of a documentdatabase such as MongoDB with those provided by software drivers, suchas “Reactive Mongo,” allowing the smart content database to quicklyrespond to requests for smart content related to an entity and outwardlystream results from such requests for such smart content in a fullyasynchronous, fully non-blocking manner in real time to a large numberof clients simultaneously. In this embodiment an intelligent monitoringagent may represent the interests of one or more users related aspecific type of content about a specific entity.

In an embodiment methods process knowledge about entities 7, whereinsuch methods find and recognize knowledge about symbolic objectsembedded in the content resources 2; process logs of user interactions 3with such symbolic objects, process user inputs 4 that contribute to theknowledge about an entity 4 and encode and store the resulting knowledgein a “knowledge graph;” also referred to herein as “knowledge graphstorage 8.

In an embodiment, methods: recognize “symbolic objects” (i.e. objectsthat represent entities, including words, phrases, codes, IDs, imagesand videos, etc.), embedded in or referred to in text files, imagefiles, video files and other types of content files from contentresources 2; compare and contrast data related to the recognizedsymbolic objects to establish confidence levels about the entitiesrecognized; and add knowledge tags to these symbolic objects, each suchknowledge tag, for example, comprised of at a minimum: a unique entityID for the entity; the name of the entity group classification for theentity (for example, “Person”); and the name of the entity type (forexample, “Physician”) classification for the entity.

In an embodiment, “knowledge” about entities, entity properties, entityrelationships and entity relationship properties is stored in the“knowledge graph storage,” 8 a graph database customized and optimizedfor storing knowledge.

In this embodiment, methods enhance and transform a standardimplementation of a commercially available graph database, such asNeo4J, into a customized knowledge graph that includes uniqueproperties, rules and methods that allow optimization of the graphdatabase to efficiently store, manage and access all types of knowledgeabout entities.

In this embodiment, all components of the knowledge graph have specificutility related to storing knowledge about entities: graph nodesrepresent entities; graph node properties represent entity properties;node relationships represent relationships among entities; andrelationship properties represent properties associated withrelationships among entities.

In this embodiment, all components of the knowledge graph have specificutility related to storing knowledge about entities: graph nodesrepresent entities; Node properties represent entity properties; Noderelationships represent relationships among entities; and relationshipproperties represent properties associated with relationships amongentities.

In an embodiment, each of the nodes of the knowledge graph areconstructed using knowledge property templates, which specify “knowledgemarkers,” i.e. a set of properties that, when considered as a whole,uniquely identify an entity and allow disambiguation among entities,each such set of knowledge markers varying according to the entity groupand the entity type of the entity, and each such set comprising atleast: an entity id, an entity name, an entity group name for the entityand an entity type name for the entity, plus other properties relevantto the entity group and entity type of the entity. For example, anentity having an entity group name “Services” and an entity type name“Restaurants,” may have a set of knowledge markers for restaurants suchas “geo-location,” “country,” “province/state,” city,” “street address,”“phone number,” “cuisine,” “reservations flag,” “delivery flag,” etc.The knowledge markers for entity types such as “attorneys,” “highschool,” “rivers,” “blogs,” “religious beliefs,” etc. will all havediffering sets of knowledge markers, that when presented as part of alist of entities to a user, will allow the user to immediately knowwhich entity is the one he or she is interested in, and subsequentlyallow a search engine to find content about only the entity of interestto the user, eliminating content related to other entities havingsimilar names, allowing a collaboration between users and machinealgorithms, and making the search process more efficient for the user.

In an embodiment, relationship among nodes of the knowledge graph areconstructed using knowledge relationship templates, comprising an entityrelationship name for the relationship between the two entitiesconnected, an entity relationship type name and entity relationshipproperty names, varying according to the set of entity groups or entitytypes connected, along with the sequence in which they are listed. Forexample, a relationship between an entity having an entity group“Person” and an entity type “User” and another entity having an entitygroup “Content” and an entity type “News Article” may have relationshipproperties specific to this specific pairing of entities such as “read”,“shared,” “annotated,” etc.

In an embodiment, smart content records in the smart content databasehave an embedded record component called a “knowledge map,” comprisingproperties aligned with their corresponding entity node in the knowledgegraph, such alignment including, for example: use of the same entity idfor the smart content record and its corresponding entity node, as wellas the same entity group name, entity type name and other propertiesspecified as knowledge markers for this entity group and entity type.For example, the smart content record for a “News Article” would have acorresponding node in the knowledge graph for an entity having an entitygroup “Content” and an entity type “News Article,” and both the smartcontent record and the node in the knowledge graph may have propertiesfor “author name,” author entity id,” “date published,” etc. The smartcontent record may also contain additional properties not included inthe knowledge graph.

In an embodiment, various methods are applied to add to and/or updatethe knowledge graph, including adding or updating entity nodes, entityproperties, entity relationships and entity relationship nodes.Additional methods are applied to add to or modify associated smartcontent records.

FIG. 2 shows a block diagram of an embodiment having methods for storingmetadata components. This metadata is used by methods for standardizingthe categorization and organization of knowledge. Such metadataincludes: metadata lists 9 containing metadata elements and metadatatables 10 linking metadata elements to other metadata elements.

In an embodiment, metadata lists 9 include: lists of classificationcategories, interaction methods, intelligent agents, resource providersand entities.

In an embodiment, metadata tables 10 link items in metadata lists 9.Examples of such linked metadata include: entity types to entity groups;entity types to entity properties; entity types to entity markers;entity types to entity interaction methods; content types to fileextensions; content types to content groups; content types to defaultentity Interaction Methods; resource property names to content objects;and standardized property names to content objects. For example, a tablemay consist of the names of properties as designated by a contentresource called “Geonames,” the names of properties as designated by acontent resource called “DBPedia,” and the standardized names ofproperties as designated in the Metadata, allowing for computerizedrationalization of content from Geonames and DBPedia, and the storing ofproperties for content from such resources in the knowledge graphdatabase and the smart content database in a normalized manner.

Metadata may be stored, for example, in a relational database such asMySQL, in order to insure the integrity of the metadata lists and mappedmetadata.

FIG. 3 shows an embodiment having methods for obtaining content fromvarious resources. Embodiments include methods for processing suchdiverse content resources 11, for example, as: RSS Feeds, tweets,messages, images, videos, computer readable files, office documents,ePublications, Web Services data, Open Linked content (“RDF Files”)data, as well as imaged print content and other types of contentresources.

FIG. 4 shows an embodiment having methods for rationalizing contentresources 4 and making them available to the Intelligent Internetsystem.

In this embodiment, methods process content files from the contentresources 2 to assign an entity ID for each unique content file 12,recognize content types for each unique content file 13, assignknowledge tags to symbolic objects in the content resources 2, creatingsmart content that is stored in the smart content database 6. In thisembodiment the knowledge tags do not act as pointers to a file having aspecific URL, as do hypertext tags supported by the World Wide Web andWeb browsers, but point instead to a node in the knowledge graph,allowing a user direct access to knowledge about tagged objects uponindication of interest by the user.

In this embodiment, methods reduce the number of file types byconverting various file types implemented in content from the contentresources 2 into file content types 13 implemented in smart contentrecords, such conversions using conversion maps defined in the metadata1. An example would be converting a .jpg or .img file type into acontent type of “image.”

In this embodiment, methods normalize content elements names 14 for dataelements in the original content, converting them into standard propertynames as implemented in the smart content database 6, using conversiontables defined in the metadata 1. An example would be converting a dataelement name “Publication date” into a standardized data element name“date published, thus simplifying downstream indexing and search methods

In this embodiment, methods normalize content data structures 16 fromthe original content in the content resources 2 into standardized datastructures in the smart content database 6, using conversion tablesdefined in the metadata 1.

In an embodiment, the knowledge stored in the knowledge graph database 8and the smart content stored in the smart content database 6 shares thesame entity ID. When a content record is added to the smart contentdatabase 6, methods add a corresponding entity node to the knowledgegraph database 8. Methods allow linking the entity node record and itscorresponding smart content record, by utilizing the same entity ID foraccess to the smart content database 6 and the knowledge database 8.Methods align the content properties 16 of these two records, forexample, the entity group, the entity type and other properties.

FIG. 5 shows a block diagram of an embodiment having methods forprocessing RSS content.

In an embodiment, methods fetch RSS summary records 20 from RSS feedresources 10 by sequentially going through a list of RSS feeds 18 knownto the metadata. Fetches of such records are governed by fetch schedulesfor RSS content 17, also known to the metadata. For example, the systemmay fetch records from a BBC Sports RSS Feed every 3 minutes, and newsfeed about a neighborhood in Wichita, Kans. every day.

In an embodiment, methods: obtain the full text of an article 21 from aproperty in the RSS XML record that provides a hypertext link to theoriginal article; obtain a list of links to images, videos and otherfiles related to the article 22; and using the links, download suchfiles, add them as records in the smart content database and as entitynodes in the knowledge graph.

In an embodiment, the standard XML properties in an original RSS summaryrecord are mapped to the entity properties in an smart content record23, such entity property names and data structures adhering to atemplate stored in the metadata 1.

In this embodiment, methods fetch records from URLs pre-defined in ametadata list of RSS feeds, each such URL in the metadata list havingpreviously: been assigned a unique entity ID; having a Node created inthe knowledge graph; and this Node having been linked in the knowledgegraph with the entity Node designating the resource. In addition, eachsuch resource having been previously assigned a unique entity ID andhaving an entity Node created in the knowledge graph.

In this embodiment, the smart content record in the smart contentdatabase 6 includes an entity ID for the RSS source content object, anentity ID for the RSS URL, and an entity ID for the RSS resource thathas published the RSS content and entity IDs for each of the authors ofthe RSS content object. In this embodiment entity nodes in the knowledgegraph database 8 are updated with relationships between the entity nodesfor each of the above-mentioned entities.

In an embodiment, methods obtain metadata tags associated with the fulltext of an article summarized in an RSS feed, such metadata includingSchema.org tags, rNews Tags and microformat tags. Methods havepreviously mapped these standard metadata tags to standard entityproperties in the metadata. Methods store the values in these metadatatags 25 in a component of the smart content record for the RSS contentobject for the article.

FIG. 6 shows a block diagram of an embodiment having methods to processRDF content.

In an embodiment methods import RDF file records 27 into a temporarystorage space and rationalize the RDF content 28.

Examples may include importing various RDF datasets, such as thoseprovided from DBPedia, a content resource 2 which stores semanticknowledge (i.e. knowledge about entity facts and entity relationships)found in Wikipedia and those provided by Geonames, a content resource 2which stores semantic knowledge about millions of places. Various suchRDF datasets may use different names for entity types. This embodimentrationalizes entity types names using a conversion table in the metadatastorage 1.

This embodiment then updates the smart content database 25, and updatesthe knowledge database 26 with the normalized content.

FIG. 7 is a block diagram showing an embodiment, having methods forprocessing content from Web Services resources.

In an embodiment functions respond to a user Interaction 30 by accessingWeb Services content resources via a REST interface 31.

Examples of Web Services users may wish to access include: YouTube,Google Search, Google Books, Yahoo Finance or any of thousands of otherWeb Services resources.

Various such Web Services may use different names for data elements theyprovide. An embodiment has methods for rationalizing the names of suchdata elements 32 from conversion tables in the metadata storage. Forexample, an image from one Web Service may have a property named by theWeb Service as “Creator” while an image from another Web Service mayhave a property named by the Web Service as “Photographer.” Thisembodiment would rationalize the names of these properties into astandardized name used by the system to simplify further access andprocessing.

This embodiment then updates the smart content database 25, and updatesthe knowledge database 26 with the normalized content.

FIG. 8 shows a block diagram of an embodiment having methods to processWeb Pages content. In an embodiment, methods respond to a userInteraction requesting access to the content on a Web Page 33.

In this embodiment, functions determine if the content on this Web pagehas been previously processed by the system 34 by accessing a list ofpreviously parsed Web pages 35. Functions determine if the current Webpage needs to be parsed and processed into smart content and knowledgeabout the contents of the Web page 34 by comparing characteristics ofpreviously parsed Web Pages 35 with smart content stored in the smartcontent database 8.

If the Web page(s) content needs to be processed, functions schedule theWeb page content for parsing 36, then parse the Web page elements 37 togenerate an HTML Dom Tree 38. Functions then analyze this content torecognize entities associated with the content 39 using any of a varietyof named entity recognition services or applications, for example, theWeb Service Open Calaise.

If the Web page(s) content has already been processed, functions getaccess to knowledge about the content using the entity ID(s) of thesmart content record(s) about the Web page(s) 40 from the knowledgegraph storage 6 and update the knowledge Store 29 to reflect the user'sinteraction with the Web page content.

FIG. 9 shows a block diagram of an embodiment having methods forprocessing office documents content.

In this embodiment, methods upload office documents; recognize dataobjects in the content from the resource; and add knowledge tags to thedata objects to create a smart content object.

In this embodiment, methods recognize and convert Word documents 41,Power Point documents 42, Excel documents 43 and PDF documents 44,Google Docs, Libre Docs and iWork Docs 45 into a standardized open fileformat, such as that supported in Office Libre; convert these files intostandardized file types, such as for example, ePub3 files 46; and storethese files as content objects in the smart content database 6.

Methods get document metadata 47 from the original content files andnormalize this metadata 48 into standard metadata elements, according tothe metadata database storage 1 and store this standardized metadata inthe smart content database 8. Methods then update the knowledge databaseto create a node for each content file, populate this node withproperties such as the entity ID for the smart content file, the contenttype for the smart content and the entity group (i.e. “Content”) for thesmart content. Methods then create relationships between this smartcontent and other entities linked to it and update the knowledgedatabase 29.

FIG. 10 shows a block diagram of an embodiment having methods forrecognizing symbolic objects and adding knowledge tags to symbolicobjects.

In an embodiment, methods recognize and create knowledge tags 49 tosymbolic objects embedded in the content object component of a smartcontent record in the smart content database, such smart objects beingprocessed from various content resources 2, including, for example,content from RSS Feeds; content from computer-readable files uploaded bya user; content from office documents uploaded by a user; and contentfrom openly accessible ePub files. Each such knowledge tag consists of anon-displayed entity id for an entity represented by the symbolic objectin the smart content record.

In an embodiment, methods recognize symbolic objects embedded in thecontent and pre-map 50 the symbolic objects embedded in the contentobject component in the smart content record, each such tag consistingof a non-displayed entity id for an entity represented by the symbolicobject in the smart content record and each such tag mapped to thesymbolic object. For example, for content coming from a Web Service“Yahoo Finance,” methods map source content elements such as the“Current price” of a stock to standard content elements as defined inthe metadata, and other methods responsive to a user indication ofinterest in the “Current price” of a stock create a tag linking thiscontent element to knowledge in the knowledge graph associated with thiscontent entity.

In an embodiment, methods create a “knowledge map” for data elements inWeb Services content and RDF content, allowing such data elements to bestored as normalized properties in a knowledge map component of a smartcontent record in the smart content database 50.

In this embodiment methods dynamically tag symbolic objects when theyappear in proprietary content, including Web pages 51. In thisembodiment upon indication of user interest in a symbolic object,methods cause a popup window to display a list of entities referred toby symbolic objects in the content, with an indication of the entity orentities matching the symbolic object of interest. For example, thecontent presented on a Web page is typically owned by the source of thecontent and legal restrictions may preclude embedding tags in thecontent object. In this case, methods receive an indication of interestby a user in a symbolic object and cause a popup to occur adjacent to oron top of the Web page content, such popup listing the entities relatedto symbolic objects on the Web page, thus allowing the user to indicatetheir interest in a symbolic object and directly access knowledge aboutthe entity which it represents.

FIG. 11 shows a block diagram of an embodiment having methods forrecognizing and processing symbolic objects embedded in text content.

In an embodiment, methods recognize symbolic objects in content havingexisting URLs 52, by sending the URL to one or more Web Services thatrecognize named entities 55. For example, during the processing ofcontent having an existing URL, methods may send the associated URL to aWeb Service such as Open Calais or Extractiv or other such named entityrecognition Web Service, or multiple such Web Services. The Web Servicedocuments the entities it finds in such content, and returns data,including, for example the names of entities it has recognized in thecontent, its confidence that it has matched the symbolic objects withthese entities and the entity types of the entities it has recognized.

In an embodiment, methods recognize symbolic objects in content nothaving existing URLs, by creating a smart content record 53 and sendingthis smart content as a text record 57 to one or more Web Services thatrecognize named entities 57.

In an embodiment, methods recognize symbolic objects in any content byprocessing this content using a commercial or open source named entityRecognition application 56 such as, for example, GATE, LingPipe,Stanford Named Entity Recognizer, Illinois Named Entity Recognizer, orany of these tools used jointly.

In an embodiment, methods get results from multiple methods forrecognizing named entities and compare and rationalize the results fromthese multiple methods 58. For example, if multiple different methodsidentify “Abraham Lincoln” as symbolic objects in a news article oreBook as representations of “Abraham Lincoln, a President of the UnitedStates, methods may determine with a high probability that this symbolicobject represent an entity known to the system and will updatecomponents of the smart content record 25 in the smart content database6 and the nodes and relationships in the knowledge database 8.

In an embodiment, when multiple methods for recognizing named entitiesproduce discrepant results 60, functions process these discrepanciesusing machine learning tools 61 and/or human analysis.

FIG. 12 shows a block diagram of an embodiment having methods forrecognizing and processing symbolic objects embedded in images content.

In an embodiment, methods implement “1” to “n” image recognition methodsand compare and contrast the results returned. When the results areuncertain, methods provide users with knowledge markers to assist theuser in deciding which entity to select. Methods allow users to link asymbolic object in an image to a specific entity.

In an embodiment, methods recognize symbolic objects representingentities embedded in images 61, such methods including multiple methodsfor recognizing people, places, cities, or facilities, such as stadiums,buildings, etc.

In an embodiment the user interacts with a device having functions tocapture/view images 62. In this embodiment, the user selects a symbolicobject in an image 63, by pointing to a region of the image on a displayof the image capture/viewing device 62.

In an embodiment, methods recognize the characteristics 65 of thesymbolic object in the region of interest selected by the user 64.Methods then match these characteristics 66 with those of images in alibrary of known symbolic objects 67. If these methods find a singlematch, they create or update a smart content record 25 in the smartcontent database 8 and update the nodes and relationships 29 in theknowledge database 6. If these methods find multiple entities matchingthe symbolic object of interest to the user, methods present a list ofpossibly matching entities to the user, along with “knowledge markers”,properties useful to help the user select among these possibly matchingentities 70. The user then indicates his or her choice of an entity forthis symbolic object 71.

Where discrepancies remain unresolved, methods perform further machinelearning to attempt to identify the entity represented by the symbolicobject of interest to the user.

An example of this embodiment would be when a user points to a person inan image containing three people in front of mountain. The user maypoint to a region of the image showing the mountain, or to a region ofthe image showing a person. Methods would then determine thecharacteristics of the symbolic object in this region and compare thesecharacteristics to a library of known symbolic objects; for example, adatabase containing previously recognized faces or previously recognizednatural landscapes If the methods determine that the reference libraryhas four faces that are similar to the one of interest to the user, itwill display these four faces, along with information about each of thefour people with faces possibly matching the face of interest to theuser.

FIG. 13 shows a block diagram of an embodiment having methods forrecognizing and processing symbolic objects embedded in video files.

In an embodiment, methods recognize a region of attention in a videoimage indicated by a user, such region of attention defined by userstouch-based gestures to circumscribe a region of attention, or by userspoint and click methods to define a region of attention.

In an embodiment, users interact with a device 73 capable of capturingand encoding a video, or capable of viewing a previously encoded video72. In this embodiment, the user selects a region of attention in avideo by touching or clicking on that region on the display screen ofthe device 73.

Methods recognize symbolic objects within a video 74, using algorithmssuch as SIFT, HMAX and/or Bayesian algorithms to determinecharacteristics of the symbolic object match these characteristics 66with the characteristics of symbolic objects found in a library of knownsymbolic objects 67.

When methods find multiple possible entities that may match the symbolicobject of interest to the user, methods provide a set of thumbnailimages of such possibly matching entities and their associated knowledgemarkers 77 to help users disambiguate among these possible entities, andselect their entity of Interest 78.

Methods get context data associated with the video from the contentresource metadata 76 and update the smart content database 25 and theupdate the knowledge graph database 29.

FIG. 14 shows a block diagram of an embodiment having methods forrecognizing and processing symbolic objects embedded in video broadcasts79.

In an embodiment, users interact 73 with a touch-responsive device 80capable of receiving and displaying a video broadcast 79, acting aloneor as a mirror to a TV attached to a network capable of transmittingmirror broadcasts to multiple devices. In this embodiment, the userselects a region of attention in a video by touching that region 73 onthe display screen of the device 80.

Methods extract characteristics from the selected region 74 andrecognize symbolic objects within the broadcast video, using algorithmssuch as SIFT, HMAX and/or Bayesian algorithms to determinecharacteristics of the symbolic object and match these characteristics75 with the characteristics of symbolic objects found in a library ofknown symbolic objects 66.

When methods find multiple possible entities that may match the symbolicobject of interest to the user, methods provide a set of thumbnailimages of such possibly matching entities and their associated knowledgemarkers 77 to help users disambiguate among the possibly matchingentities, allowing users to select their entity of Interest 78.

Methods get context data associated with the video from the contentresource metadata 76 and update the smart content database 25 and theupdate the knowledge graph database 29.

In an embodiment, methods allow a touch screen enabled smart phone,tablet or other small device 80, to tune into and receive a broadcastmirroring a broadcast being simultaneously shown on a larger screendevice.

FIG. 15 shows a block diagram of an embodiment having methods processingfor recognizing symbolic objects embedded in various types of printedcontent.

In an embodiment methods: capture an image of the printed materialscontent object using an imaging capable device 83; analyze a region ofthe printed materials image selected by the user 73 to recognize thepresence of symbolic objects 74, including text objects and imageobjects; recognize text in the printed materials image object; recognizespecific symbolic objects in the text; and associate these symbolicobjects with entities known in a library of known symbolic objects 66.

In an embodiment, examples of printed materials content objects havingmethods for recognizing symbolic objects include, for example: productpackaging 79, retail shelf tags 80, printed coupons 81, printed signs 82and other printed content materials.

In an embodiment, if an image of a symbolic object is found in theprinted materials image, methods: categorize the image into knowncharacteristics and patterns for typical types of symbols, such as barcodes, QR codes, and brand logos 74; recognize the presence of suchimages; relate these images to symbolic objects in a library of knownsymbolic objects 66; update the smart content database 25 to include arecord for this printed materials image; and update the knowledge graph29 entity nodes, entity properties, entity relationships andrelationship properties.

An example of this embodiment would be a user in a retail store usinghis or her Smart Phone to image a shelf tag. In this embodiment thesystem would recognize characteristics of the elements in the shelf tag,and relate the product designated by this shelf tag to one or moreentities known to the knowledge graph, thus allowing the user to learnabout this product/brand and further interact with it.

This has the advantage of providing users with a single consistent userexperience that allows them to not only interact with entities currentlyconnected to the Internet but also entities that are not currentlyconnected to the Internet, in this example, a passive shelf tag, or anyother kind of currently non-connected printed or otherwise visiblematerials or other content that can be imaged.

FIG. 16 is a flow chart showing an embodiment having methods forlearning explicit factual knowledge about entities and storing thisfactual knowledge as properties in a knowledge graph database node andin a component of a smart content database record.

In an embodiment, methods scan entity nodes during times of light systemresource usage to identify entity nodes with null values 84; create logsof entity nodes having missing properties; assemble a “missingproperties” list based on such scanning 85; and search for contentresources 86 by looking at metadata tables 1 mapping entity propertiesto content resources 2 to learn where to look for factual knowledgeabout such entity properties 87. Methods determine whether the contentresources have data needed to fill in the missing properties and fetchrecords from these resources for further processing 88.

In this embodiment, if resources content records contain structuredcontent properties, methods match these properties with missing entityproperties in the knowledge graph 8, and also with correspondingproperties in the smart content database 6, and then update suchproperties in the knowledge database 29 and update such properties inthe smart content database 25. If methods find conflicting possiblevalues for properties with null values, methods determine the mostlikely value by giving weight to the most-trustworthy resources 90.

An example of this embodiment would be when a method determines that anode in the knowledge graph is missing a property value for the birthdate of some celebrity or historical person, methods would scan contentresources such as The New York Times and Wikipedia. If methods findconflicting birthdates, methods would give weight to The New York Timesor some other most trusted resource, populate the appropriate propertyin the node for that person, create relationships between that node andmultiple nodes having an entity group of “Dates” and an entity type of“Birth Dates,” such relationships designating a birth date for thatperson, and each such relationship having a property designating theresource for this relationship and another property designating itsestimated trustworthiness based on machine learning.

FIG. 17 is a flow chart showing an embodiment having methods forlearning knowledge about explicit relationships among entities, andstoring this relationship knowledge in the knowledge graph.

In an embodiment, methods periodically fetch updated RDF records 91 fromRDF content Files 26 and rationalize the RDF content based on tables inthe metadata storage 1. For example, methods would periodically fetchupdated RDF records from a content source “The New York Times” whichfrequently updates its RDF database.

If the Subjects and objects in the Rationalized RDF records matchentities in the knowledge graph 92 and the predicates in therationalized RDF records match relationships in the knowledge graph 93,then do nothing. If either the subjects or objects in the rationalizedRDF records do not appear in the knowledge graph 8, then update theknowledge graph storage 34. If a predicate in the rationalized RDFrecords do not match relationship in the knowledge graph, then updatethe knowledge graph storage 34. If a predicate in the Rationalized RDFrecords is not considered a valid relationship 94 according to themetadata storage 1, add the predicate to an invalid relationshipsexception list 95 for further analysis.

FIG. 18 is a flow chart showing an embodiment having methods forlearning contextual knowledge about relationships among entities, andstoring this relationship knowledge in the knowledge graph.

In an embodiment, methods create a queue for methods to analyzecontextual entity relationships 96; fetch yet-to-be analyzed smartcontent records 97 and analyze the proximity of symbolic objects in eachsmart content record 98.

In an embodiment, methods analyze the smart content records 99 todetermine if any contextual relationships exist among symbolic objects,including relationships in which, for example: 1) symbolic objects areembedded in the same smart content record; 2) symbolic objects arepresented as parts of a list, separated by commas, within a single smartcontent record or are located in close physical proximity; or 3)symbolic objects that are images are located in close physical proximityto each other.

In an embodiment, methods determine if any symbolic objects that arecontextually linked are classified as belonging to the same entity type100, or the same entity Group 101 and methods create relationshipsbetween them.

In this embodiment, for example, if methods find two separate entitieshaving an entity type of “NASDAQ listed Companies” in close proximaterelationship in a content object having a content type of “ResearchReport,” a method may create a relationship between them 100 and assigna relationship type of “is mentioned in a Research Report along with.”

FIG. 19 is a flow chart showing an embodiment having methods formonitoring and logging user interactions with entities 106.

In an embodiment, methods monitor and log user interactions withentities, including symbolic object entities and content entities.

In an embodiment, a user may select an entity using any of a variety ofmethods 103, and then select a method for interacting with the selectedentity from among a set of valid interaction methods 104, then Interactwith this entity 105.

In an embodiment, methods log user interactions 106, and enhance thedata in these logs with user profile/context data 102.

In an embodiment, methods update user interaction statistics in acontent collection in a document dataset 107. For example, when a user“remembers” an entity “Ford F1” vehicle, methods create and thenprogressively dynamically update a statistics record for the entity“Ford F1” showing the cumulative number of times users have rememberedthis entity.

In an embodiment, methods continuously monitor and read the interactionslogs and interactions statistics and update the knowledge graph database29.

In this embodiment, for example, the knowledge graph may be updated todocument that “user A (an entity node) remembered (a relationship)entity 1 (an entity node),” or that “user A (an entity node) isfollowing (a relationship) news (an entity node) about (a relationship)entity 2 (an entity node),” or “user A (an entity node), made areservation (a relationship) at entity 3 (a Node),” etc.

FIG. 20 shows a block diagram of an embodiment having methods fororganizing and storing knowledge contributed by users.

In an embodiment, methods analyze smart content records 108 havingcontent types that, according to the metadata 1, may include records ofuser experiences, reactions, advice and other forms of knowledge relatedto an entity.

Examples of such content types include: “Travel Reviews,” “RestaurantReviews,” “Travelogues,” “Product Reviews,” “Hotel Ratings,” “Guide forselecting a hotel in San Francisco,” etc.

In an embodiment, methods determine: if the smart content is writtenfrom a first person viewpoint, or from an observational viewpoint; ifthe smart content contains one or more keywords from among of list ofkeywords in the metadata indicating a type of user contributedknowledge, for example: “review,” “experience,” or “visited.” Methodsadd new entity relationships between the author of the smart content andentities embedded in the smart content, assigning a relationship type,for example of “wrote a review referring to,” using rules based on themetadata.

FIG. 21 shows a block diagram of an embodiment having methods fororganizing components of smart content records.

In an embodiment, methods organize smart content records into smartcontent record components, including: a smart content metadata component111, a smart content knowledge map component 112, a smart contentpermissions component 113, a content summaries component 114 and a smartcontent object component 115. Methods also organize smart contentrecords into smart content summaries 110 and smart content objects 115.

In this embodiment, the smart content metadata component 111 containsmetadata about a content record provided by the content source as partof the original content record. For example, some content sources addtags to their content using the Schema.Org standard. Others use therNews standard for tagging content. Yet others tag content usingMicroformat standards. Methods rationalize these varying standards usingtables from the Intelligent Internet's metadata storage, and create arecord in the smart content storage containing these rationalizedmetadata tags.

In this embodiment, the content knowledge map component 112 containsfactual knowledge about the content stored in a template-based structurehaving data elements, including, for example: the entity ID for thecontent; the entity name (i.e. title); the entity group name (set at“content” by default); the entity type name (which varies according tothe content type); the name of the author(s) of the content; the entityIDs of the author(s) of the content; the name of the resource from whichthis content was obtained; the entity id of the resource; and timestampsfor the date and time published, date and time this content may bereleased and date and time at which this content may no longer be madeaccessible to users.

In this embodiment, the smart content permissions component 113, asdefined by the resource providing the content or by the author of thecontent, may include, for example: a flag indicating the access rightsassigned to the content, such flag denoting access rights as, “Private;”(accessible only to the resource or author(s)), “Individual” (accessibleonly by an individual user named by the resource or author(s)); “Groups”(accessible only to a group named by the resource or author(s)); alongwith lists of individuals and groups authorized to access or update thecontent.

In this embodiment, the smart content summaries component 114 may comefrom the author or resource providing the content, or may be generatedby known methods for summarizing content. A given smart content recordmay contain multiple summaries, for example it may contain a summaryconsisting of 140 characters suitable for sending text messages ortweeting, and it may also contain a summary consisting of 500 characterssuitable for displaying a summary of a news article, an email, a productpromotion or an event.

In this embodiment, the smart content object component 115 is theoriginal content obtained from the resource, enhanced by the addition ofhidden knowledge tags linking a recognized symbolic object in thecontent object with an entity in the knowledge graph.

In an embodiment, a smart content object component 115 may consist ofstructured elements, each of which can be dynamically updated by aresource. For example, a smart content object comprised of a formdisplaying stock market data about an entity, may have individual smartcontent properties updated in real time by a resource. A smart contentobject displaying structured content about a gasoline station may have asmart content property for “price per gallon or liter,” updatable by thegas station at any time. Methods can process these updates andadditional methods may generate automatic notifications to users whohave chosen to “follow” content about this entity.

The benefits of having each content record containing knowledge aboutitself are manifold. By creating a universally understandable andutilitarian structure for content up front, either at its point oforigin or as part of a large-scale repository of knowledge enhancedcontent, downstream methods that manage and use this content may operatemore efficiently. Real time analysis of “Big data” becomes possiblebecause the significant amount of “after the fact” cleaning up of dataagain and again by analysts at multiple research firms or brands may bereplaced by a single “before the fact” point of creation of “coherentcontent,” having all data aligned and normalized and having knowledgeembedded in the smart content record before such smart content isexposed to further methods and interactions.

FIG. 22 shows a block diagram of an embodiment having methods forimplementing metadata-defined elements and structures in a knowledgegraph.

In this embodiment, methods customize and optimize the standardarchitecture of an open sourced or commercially available graph databaseto explicitly store knowledge about entities.

In an embodiment, the customized knowledge graph 116 contains: nodes,each representing a unique entity; node properties, each definingfactual properties related to the entity denoted in the node; noderelationships, each representing a relationship between two entities;and node relationship properties, each defining declarative knowledgeabout the relationship between two entities.

In an embodiment, every entity node in the knowledge optimized graph hasentity properties including: entity properties universally implementedin all entity nodes, such as: node id, a unique entity id for the entitydenoted by the node, an entity group classification for the entitydenoted by a node, an entity type classification for the entity denotedby a node; and a timestamp created property denoting the data and timethis node was created.

In this embodiment: 1) entity nodes have properties conforming to theentity type of the entity represented by the node, and each such set ofproperties is derived from an entities type property template 117 forentities belonging to this entity, each such entities type propertytemplate defining specific “knowledge markers” for a specific type ofentities, i.e. properties that allow disambiguation among entities; 2)relationship between two entities conforming to a list of validrelationships 118 between the entity types for the two entities; witheach such relationship type has a relationships property template 119defining valid relationship properties for that relationship type.

FIG. 23 shows an embodiment having functions managing the creation ofentity nodes based on entity type node templates.

In an embodiment, the metadata 1 for the knowledge graph contains: alist of knowledge markers from the metadata storage 1 for each entitytype known to the system, such knowledge markers consisting of a smallnumber of specific entity properties needed to disambiguate amongentities. 120. These knowledge markers, when presented to users, allowthem to quickly perceive which entity they want to interact with, fromamong a list of similar entities. They also facilitate machineprocessing and organizing of knowledge and smart content.

In an embodiment, methods create new entity type node templates 121,including properties for each of the knowledge markers defined in themetadata storage 1 for that entity type 122. Methods set values for twouniversally required properties in each of these entity type nodetemplates: the “entity type name” and “entity group name,’ with allother properties for a given entity type node template initially set atnull. In an embodiment, methods process content from content resources,and for each content record processed create a new entity node record123 in the knowledge graph by using the entity type node template 124for the content type of the content record processed.

In this embodiment, methods assign a unique entity ID to each newcontent entity Node, and assign the same entity ID to a new associatedrecord in the smart content database, thus providing alignment andintegration between the two databases, and facilitating methods thatsubsequently index and search for content related to entities and viceversa. Methods then populate the knowledge marker properties in theentity node containing knowledge about a given content record 126.

FIG. 24 shows an embodiment having functions to optimize searching forknowledge about entities in a knowledge graph.

In an embodiment, methods optimize accessing knowledge about entities torequire from one to a few node traversals by customizing a commerciallyavailable graph database such as Neo4J.

In an embodiment, methods traverse between one entity node and anotherentity node bi-directionally, with either node being a starting pointfor the traversal.

In an embodiment, the knowledge graph contains entity nodes for eachentity group and each entity type known from the metadata 127. Anyentity group or entity type node may act as the starting point forsubsequent node traversals. This embodiment allows search methods toefficiently finding all entities having a specific entity type, forexample “News” (having an entity group of “Content” and an entity typeof “News Articles”) about “Obama” (an entity).

In this embodiment, functions for accessing knowledge about entitiesinclude: traversing the knowledge graph to find knowledge about aparticular entity, including an entity consisting of a collection ofentities, an entity consisting of content objects and an entityconsisting of collections of content objects; starting the traversalwith the “0” node, as in other graph databases; traversing to nodesrepresenting entity groups or entity types 128; then traversing to nodesrepresenting individual entities 129, each such node containing nodeproperties denoting entity properties and each such entity node beinglinked to other related entity nodes via entity relationships.

In an embodiment methods allow the encoding of many types of knowledgeinto a common structural framework, such framework consisting of acombination of entity nodes, entity node properties, entityrelationships, entity relationship properties and smart content records,Examples of types of knowledge include:

-   -   Factual knowledge required for disambiguation among entities:        may be encoded in one entity node having a few entity        properties.    -   Factual knowledge for describing entities in detail: may be        encoded in one entity node, having entity properties for the        node and one smart content record linked to the entity node via        use of a common entity id; with the entity node and the smart        content records sharing a common entity id.    -   Knowledge about relationships between entities: may be encoded        in two entity nodes having one or more relationships between the        two entity nodes.    -   Behavior knowledge: may be encoded in two entity nodes, one for        the entity denoting the actor (a user or a device) and one for        the entity denoting the thing with which the actor interacts,        having a relationship between the two nodes denoting the type of        interaction, such relationship having relationship properties        denoting unique details about the interaction.    -   Knowledge about reactions of users to entities: may be encoded        in two entity nodes, one for the entity denoting the user, one        for the entity denoting the thing which the user has reacted to,        such nodes having a relationship between them denoting the type        of reaction (voted, expressed an opinion, liked, disliked,        etc.), and such relationship having relationship properties        denoting unique details about the reaction (polarity of the        reaction, magnitude of the reaction, etc.)    -   Advisory knowledge generated by users: may be encoded in one        entity node denoting a content entity, such node having entity        properties denoting entity markers for the content and one smart        content record having a content component object containing the        advice; with the entity node and the smart content record        sharing a common entity id.    -   Procedural knowledge: may be encoded in one entity node denoting        a smart content entity, such node having entity properties        denoting entity markers for the content and one smart content        record having a content component object containing the advice;        the entity node and the smart content record sharing a common        entity id.

Embodiments of the invention have methods for encoding knowledge inwell-defined structures in a graph-based storage resource such as Neo4J,such resource customized for encoding of various types of knowledge,including such types of knowledge as: factual knowledge, knowledge aboutrelationships among entities, knowledge about the behaviors of peopleand devices, knowledge contributed by users, including attitudinalknowledge, procedural knowledge, advisory knowledge, etc.

FIG. 25 is a block diagram showing an embodiment having methods forprocessing smart messages, special types of content useful forcommunicating among users and between users and devices.

Embodiments of the invention have methods for various of types of“message resources” into one unified messaging stream of “smartmessages,” i.e. messages having knowledge about themselves and thecontent they contain, embedded in a component of each smart messagerecord. These methods include support for processing, managing,transmitting, subscribing to and receiving smart messages.

In an embodiment, methods process and rationalize many types of incomingmessages into one unified smart message database 134, having thecapability to stream 135 large numbers of smart message summaries inreal time to the message buffers of any of a variety of networkconnected devices employed by a user 136.

In this embodiment, functions organize all messages into one unifiedsystem, allowing any entity to communicate in real time with any otherentity, supporting for example: users-to-users messages 130,users-to-entities messages 131, and entities-to-users 132.

In an embodiment, functions process messages 133: to rationalize datastored in standardized components of a smart messages record stored in asmart messages database 134, and to recognize and document knowledgerelated to these messages in a knowledge database 8.

The types of smart messages may be highly diverse, and may include, forexample: emails from individual contacts; messages from individuals ingroups of Interest; messages related to groups of interest; tweets frompeople of interest in a Twitter list; tweets about entities of interestfrom any Twitter member; summaries of news articles related to entitiesof interest; deals related to entities of interest; coupons related toentities of Interest; social updates from contacts at Facebook, Twitterand LinkedIn; sports statistics for entities of interest; notices ofevents related to entities of interest; messages related to signals fromor showing the status of network connected devices and numerous othertypes of messages.

FIG. 26 is a block diagram showing an embodiment having methods forprocessing messages from diverse message sources to create a unifiedrepository of smart messages.

In an embodiment, methods allow users to generate many types of“messages” 137, including, for example: “reviews” about an eBook;“comments” about an entity, including a content entity; “notes” about anentity, including a content entity, “questions” about an entity,including a content entity; “answers to questions” about an entity,including a content entity; “requests for a price quotation” for anentity; “terms for purchasing” an entity; “requests to reserve” anentity; a “purchase order” for an entity provided by an entity;“requests for facts” about an entity; a “list of facts” about an entity;“requests for an image” about an entity; “requests for a video” about anentity; an “electronic bill,” an “electronic statement,” an“authorization to pay a bill,” a “summary of data” derived from signalsgenerated by network connected device, “instructions” for managing anetwork connected device; and other possible types of messages.

In an embodiment, methods allow suppliers to generate standard andcustomized smart messages 138 to individual users, and to groups ofusers interested in their products and/or services, including, forexample: customized “time-limited offers” for a hotel room, “specialoffers” from a restaurant to a user within a specific radius of distanceor in a particular locality, “recall notices” related to a defectiveproduct, “news” about a new product or service, a “warning” related tousing a particular credit card and many other types of messages.

In an embodiment, methods allow devices to generate and send smartmessages 139 to users and/or other devices, including for example: a“smoke alert” condition in the user's home, an “alert” indicating theunlocking of a door in the user's home, an “alert” indicating failure ofa furnace in the user's home, a “notice” about low ink in a tonercartridge (allowing the user to purchase a new toner with a singleaction) and many other types of messages.

In an embodiment, methods allow the Intelligent Internet system togenerate and send smart messages 140 to a user, including for example:the availability of new content related to an entity the user has chosento “Follow.”

In an embodiment, methods allow users to insert or associate anothersmart message or a smart content record with a smart message. Forexample a user may send an Email to another user or group of users, withhis or her comments about a news article. The message with comments, inthis example, would contain knowledge embedded in the smart messagerecord about the comment and the news article would contain knowledgeabout the smart message record about the news referred to in a smartcontent record.

In an implementation, methods process all generated messages to: 1)rationalize such messages 141 to conform to the metadata 1 and 2)process knowledge related to the smart message 142 by recognizingsymbolic objects embedded in the messages, associating such symbolicobjects and metadata with specific entities 143 and 3) documentknowledge about such symbolic objects by adding entity nodes and entityrelationships to the knowledge graph 8.

In an implementation, methods store smart messages in entity nodes,entity node properties, node relationships and node relationshipproperties in the smart messages database 148, and update properties incomponents of the smart content database 6.

In an implementation, methods create “entity mailboxes” 146 containinglinks between a specific entity and smart messages related to thisentity. In this implementation, methods push “smart message summaries”to “smart message buffer” in any of the device registered to a specificuser 147 interested in or subscribing to entities.

FIG. 27 is a block diagram showing an implementation having methods forpartitioning smart messages into smart message components.

In an implementation, methods manage the storage of smart messagerecords 147 by separating such smart messages into smart messagecomponents, each containing the entity id for the message, suchcomponents, including for example: a smart message metadata component149, a smart message permissions component 150, a smart messageknowledge map component 151, a smart message summaries component 152 anda smart message object component 155.

In this embodiment, the smart messages metadata component 149 containsmetadata about a smart messages record provided at the time of thecreation of the messages record by its author or generated by automatedmethods during its conversion into a smart message, such metadataincluding for example: the name(s) of the author(s) of the smart messageand the entity ids of the author(s) of the smart message.

In this embodiment, the smart content permissions component 150, asdefined by the author of the content, may include, for example: one ormore flags indicating the access rights assigned by the author(s) to thesmart message, such flags denoting access rights as, “Private;”(accessible only to the author(s)), “Individuals” (accessible only byone or more designated individuals named by the author(s)); “Groups”(accessible only to a group named by the author(s)), along with lists ofindividuals and groups authorized to access or update the smart messageand “Public’ (accessible by all users).

In this embodiment, methods create a smart message knowledge mapcomponent 151 containing knowledge about the smart message, with data inthe knowledge map component stored in a template-based structure havingknowledge-related data elements, including, for example: the entity idfor the content; the entity name (i.e. the title for the smart message);the entity group name for the entity (set at “Content” by default); theentity type name for the entity (which varies according to the messagetype); the name(s) of the author(s) of the smart message; the entity idsof the author(s) of the smart message; and timestamps for the date andtime of the sending of the smart message and date and time at which thissmart message may no longer be accessible.

In this embodiment, methods create a smart messages summaries component152, using a summary coming from the author of the smart messagecontent, or methods for automatically summarizing content. A given smartmessage record may contain multiple message summaries, for example asummary consisting of 140 characters suitable for sending text messagesor tweeting, a summary consisting of 500 characters suitable fordisplaying more of the smart message.

In this embodiment, methods create a smart message object component 153,having the original message created by its author, enhanced by theaddition of hidden knowledge tags associated with recognized symbolicobjects embedded in the smart message content object.

In an embodiment, a smart message content object component 153 mayconsist of structured elements, each of which can be dynamically updatedby a content resource. For example, a smart message object componentcomprised of a form displaying stock market data about an entity mayhave individual smart content properties updated in real time by acontent resource. A smart message object displaying structured contentabout gas at a gasoline station may have a smart content property for“Price per Gallon,” updatable by the gas station or a central service atany time.

The benefits of having each smart message record contain knowledge aboutitself are manifold. By creating a universally understandable andutilitarian structure for each type of message up front, either at itspoint of origin or produced soon thereafter by automated methods as partof pre-processing the message for storage in a large-scale repository ofsmart messages, subsequent methods that manage, use and distribute thesesmart messages may operate more efficiently.

In this implementation, access to smart messages is optimized to allowusers to view a message summary from a storage buffer in the user'sdevice or from a networked storage device. When requested by the user,methods retrieve the full message.

FIG. 28 is a block diagram showing an embodiment having methods forstoring smart message summaries in an iPv6-addressable storage system,with one IPv6 assigned per entity.

In this implementation, methods assign a unique IPv6 address 156 foreach entity known to the knowledge graph. Methods store smart messagesummaries, including availability status flags 157, time-createdtimestamps 157, the content type 159 and the name(s) of the author(s) orsource of the smart message 160 in a smart message store having datastructures optimized for both indexed and sequential reading of suchmessage summaries.

FIG. 29 is a block diagram showing methods for implementing an alternateembodiment storing smart message summaries in an URI-based system.

In this implementation, methods assign a unique URL stub address 159 foreach entity known to the knowledge graph. Methods store smart messagesummaries, including availability status flags 155, time-createdtimestamps 156, the message content type 157 and the name of theauthor(s) or source of the smart message 158 in a smart message storehaving data structures optimized for both indexed and sequential readingof such smart message summaries.

FIG. 30 is a block diagram showing an embodiment having methods forsubscribing to smart messages.

In an embodiment, methods allow users to: subscribe to 1) smart messagesummaries about a single type of content about a single entity 161; 2)smart message summaries about multiple types of content about a singleentity 162; subscribe to all types of content about a single entity;subscribe to a single type of content about all entities 163; and anyother set of content types and entities, or content types and entitytypes or content types and entity groups with each intersection of acontent type and an entity constituting a “knowledge cell” that methodscan index, search and present to users.

In an embodiment, methods index use a commercial or open source tool,such as Lucene, or Apache SOLR, which uses the Lucene library, to indexsmart content in a document database such as MongoDB, as well asknowledge about entities in a graph database such as Neo4J, to createsingle or multiple federated indexes, such indexes having capabilitiesfor example, to manage full-text search, highlighting of indexed terms,faceted search, dynamic clustering, database integration, and richdocument handling.

In and embodiment, methods create “Knowledge Cells,” each of which iscomprised of “Content Particles” having: 1) a specific type of content2) about a specific entity, with each knowledge cell indexed by contenttype and by entity.

This architecture facilitates the ability of a user to follow “news” (acontent type) about “Obama” (an entity).

FIG. 31 shows a block diagram showing an embodiment having methods to“Follow” smart messages about any entity.

In an embodiment, methods allow a user to view and interact with smartmessages about any entity 164.

In an embodiment, methods request to receive a continuous stream ofsmart messages related to a selected entity; manage the organization andcomposition of this inbound message Stream 165; and manage the storageof the inbound smart message summaries in a real time in a smartmessages buffer of device(s) associated with a user 167.

Methods allow “push-based” delivery of a continuous smart message streamto each of the user's registered devices, allowing the user to receivein real time or near real time, alerts about new smart content relatedto, for example: “Ford Motor Company,” or more specifically about “News”(a specific content type) related to “Ford Motor Company.” Otherexamples include allowing the user to subscribe to tweets (a specificcontent type) regarding a job candidate, or about new deals about anentity type classified as “digital cameras.” This ability allows theuser to triangulate on a specific type or types of content about aspecific entity, allowing the user a single unified user experience forlearning about new smart content relevant to entities of interest tothem.

FIG. 32 is a block diagram showing an embodiment having methods allowingsuppliers of commodities, goods, products and services to communicateone-to-one in real time with “opt-in” users explicitly interested inwhat they may have to offer.

In an embodiment, methods allow bi-directional real time communicationsbetween users and suppliers, such communications requiring the user orsupplier to request such communications with its counterpart(s) 168, andthe user or supplier to authorize such communications with itscounterpart(s) 169.

In this embodiment, methods allow either party to interact directly withthe other party 170, 171 by sending unstructured smart messages,structured smart messages or a combination of structured andunstructured smart messages, examples including: requests from a userfor a specific type of content about an entity; requests for deal termsin a structured format from a supplier, supplemented by personalizednotes; and indications of interest in buying or offering commodities,goods, products or services.

In one example of such an embodiment, methods may allow a user, or anintelligent agent acting on behalf of a user, to publish a request for ahotel room for a particular date or span of dates, and providestructured parameters indicating the user's preferences andrequirements. In this embodiment methods allow a hotel manager toreceive requests from users and match user parameters for renting ahotel room on a specified date or span of dates with their inventory ofrooms. Methods may also allow the hotel manager to model the risksassociated for making a particular price offer, and anonymously issue anoffer to the user, along with basic details about the room and the hotelmaking the offer. Methods may: allow both parties to remain anonymousduring this process; allow the user to accept or reject the offer; andallow the user to finalize the transaction in one step, including payinga deposit, or paying for the room.

In an embodiment, methods would allow users, seeking any kind ofcommodities, goods, products or services to send smart messages tosuppliers having inventories that match characteristics of what a userseeks; for a supplier to receive messages only from qualified users; andfor a supplier to respond with personalized offers to the user. Thisembodiment thus creates a personalized one-to-one virtual purchasingsystem for the user and a highly efficient and cost effectivepersonalized one-to-one marketing system for the supplier.

In various embodiments, the one-to-one marketing interaction paradigmmay replace a wide array of existing non-personalized eCommerce systemswith personalized interactions that meet the real time needs of usersand the real time inventories of suppliers.

FIG. 33 shows a block diagram of an embodiment having methods forselecting entities for interactions.

In this embodiment, methods support: selecting an entity for interaction172 via 1) searching for entities using key words, assisted by entityknowledge markers 173; 2) selecting a symbolic object representing anentity from among a list of symbolic objects 174; 3) selecting asymbolic object representing an entity from among a gallery of symbolicobjects 175; 4) selecting a symbolic object representing an entityembedded in smart text content 176; or 5) selecting a symbolic objectrepresenting an entity embedded in untagged content, including textcontent, image content, images of printed materials and images capturedfrom videos 177.

Methods allow users to select entities for interaction using multiplestrategies; giving users the freedom to get what they want how, when andwhere they want it.

FIG. 34 is a block diagram showing an embodiment having methods tosearch for entities, assisted by entity knowledge markers.

In this embodiment, methods for detecting an indication of a userinitiated search and activating an “Intelligent Search Agent.” Methodssupport analyzing the incoming stream of characters from a search 178;communicating with a knowledge graph to request a list of entitiespotentially match the search arguments in the search request; requestingand displaying knowledge markers (properties for disambiguating amongentities) associated with those entities 180 organized by entity Groups179 and entity types 180 to help the user quickly select what they want;and receive a list of valid interaction methods for an entity selectedfrom the list. Such methods serve a similar purpose to methods employedfor disambiguating among people based on a small set of specificfingerprint “markers” and other methods employed for disambiguating anddiscriminating among people and among species of living things based ona small set of “DNA markers.” Employing methods to discriminate amongsymbolic objects and relate them to specific entities using a small setof “knowledge markers” consisting of specific comparable properties mayimprove up-front disambiguation and indexing of symbolic objects,thereby reducing the potential for the “false” matches that may bereturned by many search engines currently employed.

In an embodiment, methods support progressively narrowing the list ofsuch entities upon receipt of further characters in a search argument.

If the Intelligent Search Agent receives more than one potential result,it opens a panel in the user interface; displays a list of potentialentities and their knowledge markers 180; organizes the list by theentity group 179 and the entity type 180 of the potential entities.

Methods support monitoring inputs received by the user interface, andupon receiving an indication that the user has selected a specificentity, opening a user interface panel showing a menu listing validinteraction options for the selected entity.

Methods support “crowd understanding” of symbolic objects, by takinginto account the number of tunes users associate a specific symbolicobject having specific characteristics with a specific entity, as wellas the levels of expertise of users making such associations. Additionalmethods may then highlight or otherwise indicate that a specific entityis the most likely match for a specific symbolic object indicated to beof interest to a user.

Current methods for searching Web content and many other types ofcontent index such content using Lucene or another such indexing tool toallow searching for specific character strings. These methods returncontent that may or may not have relevance to the user. In anembodiment, methods provide instant access to knowledge about specificentities, allowing the user to precisely specify which of these entitieshe or she wishes to interact with. This collaboration between humans andmachines provides greater control by users, and may reduce time spentnavigating among various Web pages by a user to find what he or shewants.

FIG. 35 is a block diagram showing an embodiment having methods forselecting an entity of interest from among a list of symbolic objects.

In an embodiment, methods support displaying a list of symbolic objectsin a panel in the user interface 184, and detecting an indication thatthe user has selected a specific symbolic object 185 from this list 153.

In this embodiment, each item in the list of symbolic objects isassociated with a unique entity, along with its entity group and entitytype. If the entity is a content entity each item in the list ofsymbolic objects is associated with the content type of the entity.

In an example of this embodiment the system may display a list of topicsfor browsing news content, each such topic being a symbolic objecteasily understandable by users. When the user selects one of thesetopics, methods then retrieve news content related to the selectedentity.

FIG. 36 is a block diagram showing an embodiment having methods forselecting an entity from among a gallery of symbolic objects.

In this embodiment, methods support: displaying a gallery of visualsymbolic objects in the user interface 185 and detecting an indicationthat the user has selected a symbolic object from this gallery 184.

In this embodiment, each item in the gallery of symbolic objects, forexample a “thumbnail image” of the cover of an eBook, is associated witha unique entity (i.e. an eBook), along with its entity group and entitytype.

In an embodiment, methods support placing a consistent icon 186 to flagan entity with which the user may interact, adjacent to or within eachsymbolic object in the gallery, such icon being responsive to userindications of interest, and any such indication of interest triggeringmethods that retrieve and display a list of valid methods forinteracting with the entity of interest to the user.

FIG. 37 is a block diagram showing an embodiment having methods forselecting a tagged symbolic object embedded in smart text content.

In an embodiment, methods support displaying smart content objectsconsisting of text, with each such textual symbolic content objecthaving one or more embedded knowledge-tagged symbolic objects. Methodssupport detecting an indication that the user has selected a specifichidden knowledge-tagged symbolic object 187. The knowledge-tag 188associated with the symbolic object consists of an entity id for theentity represented by the symbolic object, along with the entity groupand entity type for the entity.

FIG. 38 is a block diagram showing an embodiment having methods forselecting an untagged symbolic object embedded in smart text content.

In an embodiment, methods support displaying symbolic objects consistingof text, such symbolic objects having untagged symbolic objects;detecting an indication the user has selected a physical region of thecontent object 189; and activating symbolic object recognition methods.If the symbolic object recognition methods recognize a symbolic objectinside the physical region 190, methods request knowledge about thesymbolic object from the knowledge graph, including the entity id of theentity represented by this symbolic object, and its entity group andentity type.

FIG. 39 is a block diagram showing an embodiment having methods forselecting an untagged symbolic object embedded in an image object.

In an embodiment, methods support displaying an image object, havinguntagged symbolic objects embedded within it; detecting an indicationthe user has selected 191 a physical region of the image object 192; andactivating symbolic object image recognition methods. If the symbolicobject image recognition methods recognize a symbolic object inside theindicated physical region, methods request further knowledge from theknowledge graph, including the entity id of the entity associated withthis symbolic object, and its entity group and entity type.

FIG. 40 a block diagram showing an embodiment having methods forselecting an untagged symbolic object embedded in printed content.

In an embodiment, methods support: capturing an image of a printedcontent object, such image having embedded untagged symbolic objects194; detecting an indication the user has selected a physical region ofthe printed content image object 193 and activating symbolic objectimage recognition methods.

If the symbolic object recognition methods recognize a symbolic objectinside the indicated physical region, methods support requestingknowledge from the knowledge graph, including the entity id of theentity associated with this symbolic object, and its entity group andentity type.

FIG. 41 is a block diagram showing an embodiment having methods forselecting an untagged symbolic object embedded in video content.

In an embodiment, methods support monitoring the user interface of avideo player for indications of interest from users while viewing anuntagged symbolic object embedded in the video 196, such indicationsoccurring when a user touches a region of interest in the video playerdisplay 195. Upon such indication, methods support capturing an imagefrom the video content object.

Methods support detecting an indication the user has selected a physicalregion of the video object; activating symbolic object image recognitionmethods; recognizing a symbolic object inside the indicated physicalregion; requesting knowledge from the knowledge graph, including theentity ID of the entity associated with this symbolic object, and itsentity group and entity type.

FIG. 42 is a flow chart showing an implementation having methods foraccessing knowledge about valid interactions for entities.

In an embodiment, methods support receiving an indication that a userhas selected or referenced a knowledge-tagged symbolic objectrepresenting an entity 175, and activating an Intelligent InteractionAgent upon such an indication 197.

In this embodiment, if the indication is about selection of aknowledge-tagged symbolic object, the Intelligent Interaction Agentrequests knowledge from the knowledge graph about currently validinteraction methods for this entity 200.

In this embodiment, if the indication is about a user selection of anuntagged symbolic object, the Intelligent Interaction Agent initiatesmethods to determine the characteristics of the region of attention 198,including its relative physical location within an image or video, andattempts to recognize the symbolic object and relate it to an entityknown in a library containing the characteristics of known symbolicobjects 66.

In an embodiment, functions display possible entities that may match thesymbolic object displayed to the user and respond to indications fromthe user selecting one of the entities 199.

In this embodiment, the Intelligent Interaction Agent implementsfunctions to display to the user a list of possible valid Interactionsfor the selected entity 201, such valid interactions being known to theknowledge graph. Upon receiving an indication, either from a responsivedisplay or from spoken terms, that the user has selected one of theseinteraction methods, the Intelligent Interaction Agent activates a SmartInteraction Module, which the user can utilize to interact with anentity of interest.

In an embodiment, functions performed by the Intelligent InteractionAgent are implemented locally on the user's device in JavaScript, or ina function implemented in a computer programming language understood bythe operating system of the user's device.

FIG. 43 is a block diagram showing an embodiment having methods tomanage user interactions with entities, initiated by the user selectingan entity.

In an embodiment, the user first selects an entity 203 using any of avariety of methods. An Intelligent Interaction Agent presents the userwith a menu list of valid ways to interact with this entity, with itemsin the menu limited only to interactions valid for this entity, suchvalid interactions being appropriate from the user's specific location,at a specific point in time. The Intelligent Interaction Agent gathersand correlates inputs from the user, the user's device and the knowledgegraph, then presents the interaction options to the user. The user thenselects one of the valid interaction methods 204 and interacts withcontent and other resources related to the entity 205.

In an example of this embodiment, the user may select a symbolic objectrepresenting a restaurant, the Intelligent Interaction Agent willrequest data about the user's location from a function residing in theuser's device and request a list of currently valid interaction methodsfor this restaurant from the knowledge graph. If the restaurant has aninteractive reservation system, in this example, the IntelligentInteraction Agent will display “Make a Reservation” as a validinteraction method in the user interface. If the restaurant deliversfood or provides food pickups, it may show “Order Delivery” or “OrderPickup” in the user interface.

In another example of this embodiment, the user may select a symbolicobject representing a NASDAQ Company; the Intelligent Agent will display“Market data” as an interaction option in the user Interface, and set upa connection to a Web Service, for example, the Yahoo Finance Marketdata Web Service. Methods allow for automated interactions without theneed for step-by-step user inputs.

FIG. 44 is a block diagram showing an embodiment having methods tomanage user interactions with entities, initiated by the user selectingan interaction method.

In an example of this embodiment, the user may enter or select aninteraction called “Remember anything,” and then input a phone number inan entry box. The Intelligent Agent will recognize this input as a phonenumber and ask details about this phone number, such as the name of theperson or company having this phone number. The user may then specifythat this number is for Bill Johnson's cell phone, and the IntelligentAgent will then create an entity node for this phone number. If theKnowledge Graph does not know about Bill Johnson, the intelligent agentwill ask the user to supply additional details, and then create anentity node for Bill Johnson and link this node to the phone numbernode. It will then create a relationship between the entity noderepresenting the user and an entity node representing Bill Johnson.

FIG. 45 is a decision matrix showing an embodiment having methods formanaging user interactions with entities.

In an embodiment, the metadata has knowledge about valid interactionmethods 210 that may apply universally to all entities 206, suchuniversal interactions including, for example: “Remember” [an entity],“Follow” [an entity], “Make a note” about [an entity], “Ask a question”about [an entity] or any of many other such interactions with universalutility.

In an embodiment, the metadata has knowledge that that certain validinteraction methods 211 apply only to entities belong to one or moreentity groups 207, for example: “Locate” [an entity] can only apply topeople, places, facilities, etc., (i.e. entities having a physicalembodiment at one or more locations.)

In an embodiment, the knowledge graph has knowledge that certain validentity type interactions 212 apply to entities belonging to one or morespecific entity types 208.

In an embodiment the knowledge graph has knowledge that certain specificinteraction methods 213 apply to entities having specific entityproperties and/or relationships with other entities 209, for example: anentity denoting a restaurant may or may not support reservations, a factthat may be known to the knowledge graph.

FIG. 46 is an illustration showing an embodiment having methods forreceiving an indication of interest in an entity by a user based on theuser touching or clicking on an icon to select a smart content recordfor Interaction.

In an embodiment, methods allow users to indicate an interest in contententities 214 by touching or clicking on an icon 215. The icon may belocated next to the title of the content 216 or, if an image, adjacentto the image, or even embedded in the image.

FIG. 47 is a block diagram showing examples of an implementation havingmethods for interacting with varying types of smart content.

The term “content” denotes an entity that provides information about oneor more symbolic objects. The term “smart content” denotes contenthaving a record with at least two components, a component containing the“source content” and a component having a “knowledge map” comprised ofidentifiers for one or more entities related to the content,relationships between the entities and the content and relationshipsbetween the entities and other entities.

In an embodiment, each content object has an entity group (for content,the entity group is “Content”) 217 and an entity type (also called a“content type,” an example being a “Blog”) 218. A metadata table definesa default file type (for example: .txt) for each content type (forexample: “article”).

In an embodiment, some original content from resources is comprised ofstructured content elements.

In such an embodiment, methods access content from content resources,such source content having content elements named by the resource.Methods store the names of these content elements in a data dictionaryin the metadata and map them to standard entity properties defined inthe metadata database. Methods compare the data type for each suchsource content element with data in the metadata and map the sourcecontent elements to data types implemented in the knowledge graphdatabase and the smart content database. An example would be acontent-object from YouTube, comprised of content elements that includethe name of a video, the name of the creator, the date of creation, etc.In an embodiment, methods normalize the names and data structuresimplemented by YouTube to content properties defined in the metadata 1.

In an embodiment, some content objects are comprised of unstructuredcontent, an example being a content type named “Articles” or a contenttyped named “Jokes.”

In an embodiment, methods link each content object with a default SmartInteraction Method 221 applicable to that content type, with the defaultSmart Interaction Method varying according to the content type. Forexample, an eBook may have a default Smart Interaction Method of “Readan eBook” while a video may have a default interaction method of “Play avideo.” Selecting a content object activates an Intelligent InteractionAssistant that displays possible interactions with this content objectand defaults to the default Smart Interaction Method.

In an embodiment, methods manage all aspects of a user's interactionswith content of interest. In this embodiment, users may touch or selectany content object they see in the user interface and “Remember” it, aninteraction that adds this content to their personal “Memory Bank,” acloud-based service containing everything a user wishes to remember,organize and access at any time, including: articles; tweets; websites;images; videos; eBooks; contact cards; messages; research reports;reviews about a product, or indeed, any type of content about any typeof product or service; facts, such as the VIN number of a car; eventssuch as birthdays quotations; jokes; notes about anything; commentsabout anything; reactions, ratings and reviews for any type ofcommodity, product, service, place or event; records about anything,such as an employment record; lists of people, including current andformer classmates, current and former work colleagues; current andformer customers; lists of services, including hotels, restaurants,spas, golf courses and much more.

In this embodiment, methods continuously update content properties asthe Intelligent Internet system learns about changes to the smartcontent database, for example, a restaurant may change its hours ofservice and post this change to the smart content database, and any userhaving that restaurant in their personal memory bank will have access tothe updated properties at any time.

FIG. 48 is a block diagram showing an implementation having methods forinteracting with entities using “Smart Interaction Modules.” (‘SIMs”)

In an embodiment, each Smart Interaction Module is implemented in anintegrated 3-level Model-View-Controller (“MVC”) structure.

In this embodiment the Model is based on the entity propertiesimplemented in the knowledge graph and the smart content database; the“view” 222 consists of CSS and HTML software code, designed for use witha templating engine such as “Handlebars,” and built utilizing adevelopment framework such as angular.js 223; and the “controller”functions 224 are implemented in JavaScript or a language understood bythe operating system for the user's device and packaged into functionslibraries 225. The data model 226 is defined in the metadata in the formof tables defining data model templates for each Smart InteractionMethod 227. The methods implementing the model, view and controller arepackaged as complete “Smart Interaction Modules” or “SIMs” 228, havingmethods for performing create, read, update and delete transactions witha knowledge graph database 8, a smart content database 6 and a smartmessages database 136.

FIG. 49 illustrates a design showing an implementation having methodsfor managing an “Adaptive User Interface” responsive to user selections,user contexts and inputs from network-connected devices.

In this implementation, methods allow users to select an “activity mode”229, comprised of a set of related activities, allowing the user tooperate, for example: in a “Personal” mode, a “School” mode, “Work”mode, “Family” mode, “Research” mode,” etc.

In this implementation, methods allow users to select a specific“activity” 230, comprised of a single, focused set of interactions, forexample: “Read News,” “Read an eBook,” “Watch TV,” “Manage Travel,”“Shop for Products,” “Shop for Service,” etc.

After selecting a particular activity, in this implementation, methodsallow users to select particular “activity options,” 231 for example:while reading news, the user may select to browse news topics, andselect a broad topic such as “sports,” or a more narrow topic such as“Dallas Cowboys.”

Once focused on a particular activity option, in this implementation,methods allow the user to select among various “interaction options,”232 for example: upon selecting to read news about the “Dallas Cowboys,”the user may “browse news summaries” about the Dallas Cowboys, “browsevideos” about the Dallas Cowboys, or undertake any valid method forinteracting with the Dallas Cowboys.

Finally, in this implementation, methods allow the user to “Interact”with an entity of interest within a dedicated “Interaction Panel” 23.

FIG. 50 illustrates a design showing an example of an implementationhaving methods for managing an adaptive user Interface on a singlevisual “page.”

In this example a “header panel” 234 consists of a visual containerdisplaying elements, including for example, a “user search term inputbox” 178, allowing the user to indicate a word or term of interestrepresenting an entity with which the user would like to interact.

In this example, a visual container referred to as a “header optionspanel” 235 contains controls allowing the user, for example, to: selectan “activity mode” 229, indicating the user's focus on a related set ofactivities; view a “history” of prior interactions,” “manage routines”of frequently repeated sequences of interactions, etc.

In this example, a visual container referred to as an “activities” panel237 containing options allowing the user to select among specificactivities such as, for example: “News,” “Images,” “Videos,” “eBooks,”“TV,” “Office Documents,” “ Travel,” “Shopping,” “Services,” etc.

Having selected “News,” in this example, a visual container referred toas an “activities options” panel 238 displays a pane showing options forreading news, for example: “My News,” “Health,” “Politics,” “Sports,”“Technology,” etc.

In this example, a visual container referred to as a “universalinteractions” panel 239 contains options allowing the user to indicatehis or her desire to seek help in interacting with something. Examplesof universal interactions may include, for example: “Remember anything,”“Follow anything,” “Share anything,” “Ask a question about anything,”“Record anything,” and other such interactions people commonly may seekhelp performing.

For this example, a user may touch “Remember anything” on the screen ofhis or her Smart Phone, then enter a phone number, or the name of aperson, or drag something from the interactions panel container (forinstance, a paragraph from a news article) onto the “Remember anything”box. An intelligent agent will recognize the characteristics of what theuser wishes to remember (for instance, it will recognize a phone number)and interact with the user to determine further details about the entitythe user wishes to remember, then interact with the knowledge graphdatabase to create a node for this phone number and link it to a personor company and interact with the smart content database to create aproperty for an smart content record having properties about an entity.

FIG. 51 illustrates a logical flow chart showing an implementationhaving methods for managing an adaptive user interface responsive touser selection of a universal interaction.

In this embodiment, the UI contains a panel having options forinteracting with “things” 239, i.e. entities of any type. Users maychoose among such options as: “Remember anything,” “Follow anything,”“Share anything,” “Record anything,” “Ask a question about anything,”along with other universal options. Upon receiving an indication ofinterest in any of these universal interactions, methods pop up a panelhaving panes for each type of universal interaction, such panescontaining interaction specification forms 243 allowing a user to selector specify the entity with which they wish to interact 244, and specifyhow and when they wish to interact with this entity 245. After providingthe necessary details about the requested interaction, the user thenindicates his or her decision to activate the interaction 246. Methodsthen implement the interaction 247.

Today, various websites such as Facebook, Amazon, Yelp and others allowusers to indicate that they “like” something or provide their “rating”or “review” about something. These methods for interacting with thingsare generally specific and limited to a unique thing and limited type ofinteraction, and require operators of Websites participating in thesemethods to embed customized code for specific “widgets” from specificaggregators of interaction data into their Web page software code. Incontrast, methods in the present invention allow users to interactuniversally with any entity they can see, hear or specify and specifytheir personal requirements for such interactions.

Various exemplary embodiments are described herein. Reference is made tothese examples in a non-limiting sense. They are provided to illustratemore broadly applicable aspects of the disclosed technology. Variouschanges may be made and equivalents may be substituted without departingfrom the true spirit and scope of the various embodiments. In addition,many modifications may be made to adapt a particular situation, process,process act(s) or step(s) to the objective(s), spirit or scope of thevarious embodiments. Further, as will be appreciated by those with skillin the art, each of the individual variations described and illustratedherein has discrete components and features that may be readilyseparated from or combined with the features of any of the other severalembodiments without departing from the scope or spirit of the variousembodiments. All such modifications are intended to be within the scopeof claims associated with this disclosure.

We claim:
 1. A computer-implemented method for acquiring and storingknowledge about entities, the method comprising: storing metadata aboutentities in a metadata database, the metadata database comprising listsof entities and tables mapping relationships between the entities;storing knowledge about the entities and relationships between theentities in a knowledge graph database, wherein the knowledge graphdatabase comprises factual knowledge data about entities, relationshipknowledge data between entities, and behavioral knowledge data betweenentities; storing smart content records about each of the entities in asmart content database, each of the smart content records having acomponent comprising the source of the content and a componentcomprising knowledge about the content in the smart content record; and.providing knowledge to a user useful for interacting with entities.